Skip to content

Reset configSelector on realChannel while entering IDLE state#12832

Open
kkalin68 wants to merge 3 commits into
grpc:masterfrom
kkalin68:xds-fix-channel-idle-limbo
Open

Reset configSelector on realChannel while entering IDLE state#12832
kkalin68 wants to merge 3 commits into
grpc:masterfrom
kkalin68:xds-fix-channel-idle-limbo

Conversation

@kkalin68

@kkalin68 kkalin68 commented May 27, 2026 •
edited
Loading

Copy link
Copy Markdown

ManagedChannel will stuck in IDLE state when xDS control plane doesn't have a resource anymore.
The scenario is following:

  1. Channel is open for a xds resource.
  2. XdsNameResolver subscribes to the resource on xDS control plane.
  3. The resource is removed from xDS control plane for extended period of time (unhealthy for more than the idle timeout on Channel)
  4. Channel enters into TRANSIENT_FAILURE
  5. The Idle timeout triggers and Channel shutdowns XdsNameResolver and other resources. xDS watchers are removed.
  6. The resource comes back online on xDS control plane.
  7. A new GRPC call is executed targeting the channel.
  8. The channel stays in IDLE state and reports: "io.grpc.StatusRuntimeException: UNAVAILABLE: LDS resource xxxx does not exist nodeID: yyyy" because realChannel.configSelector still points to old state.
public <ReqT, RespT> ClientCall<ReqT, RespT> newCall(
        MethodDescriptor<ReqT, RespT> method, CallOptions callOptions) {
      if (configSelector.get() != INITIAL_PENDING_SELECTOR) {
        return newClientCall(method, callOptions);
      }
...

@linux-foundation-easycla

linux-foundation-easycla Bot commented May 27, 2026 •
edited
Loading

Copy link
Copy Markdown

CLA Signed
The committers listed above are authorized under a signed CLA.

  • ✅ login: kkalin68 / name: Konstantin Kalin (f09b7d9)

@AgraVator

AgraVator commented Jun 2, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Hey,
Please fix the failing test cases. Thanks.

@AgraVator
AgraVator requested review from Copilot June 2, 2026 02:58
Copilot AI reviewed Jun 2, 2026
View reviewed changes

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes a ManagedChannel idle-mode edge case where a stale InternalConfigSelector can prevent the channel from properly exiting IDLE after xDS resources disappear and later reappear.

Changes:

  • Reset realChannel’s config selector to INITIAL_PENDING_SELECTOR when entering IDLE, forcing the next call to trigger resolver/LB restart.
  • Prevent pending-call reprocessing when updateConfigSelector() is called with INITIAL_PENDING_SELECTOR (only reprocess when transitioning away from the initial pending selector).

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread core/src/main/java/io/grpc/internal/ManagedChannelImpl.java
@kkalin68

kkalin68 commented Jun 2, 2026

Copy link
Copy Markdown
Author

Hey, Please fix the failing test cases. Thanks.

The tests are passing locally. Also the failures in tests(11) and tests(17) runs are unrelated to changes I made. And same tests passed in tests(8) and test(21). Also I don't see an option to retrigger the failed tests.

@kkalin68

kkalin68 commented Jun 8, 2026

Copy link
Copy Markdown
Author

Is it anything else I should do to get the change reviewed ? We face this issue occasionally in production and it requires a service restart to reset GRPC channel

@kannanjgithub

kannanjgithub commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

Is it anything else I should do to get the change reviewed ? We face this issue occasionally in production and it requires a service restart to reset GRPC channel

Can you address the Copilot review comment above for adding a new test?

@kkalin68

kkalin68 commented Jul 10, 2026

Copy link
Copy Markdown
Author

Is it anything else I should do to get the change reviewed ? We face this issue occasionally in production and it requires a service restart to reset GRPC channel

Can you address the Copilot review comment above for adding a new test?

I added tests. Please note, I used Copilot to navigate and create the tests

@kkalin68

kkalin68 commented Jul 10, 2026

Copy link
Copy Markdown
Author

Failed test(11) is not related to my changes. Same test passed for other Java versions

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot Awaiting requested review from Copilot

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@kkalin68 @AgraVator @kannanjgithub